{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reinforcement Learning (RL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In RL, there is an **agent** and the **environment**. The environment is the world that the agent lives in and interacts with. \n",
    "<br><br>\n",
    "\"At every step of interaction, the agent sees a (possibly partial) observation of the state of the world, and then decides on an action to take. The environment changes when the agent acts on it, but may also change on its own.\n",
    "\n",
    "The agent also perceives a **reward** signal from the environment, a number that tells it how good or bad the current world state is. The goal of the agent is to maximize its cumulative reward, called **return**. Reinforcement learning methods are ways that the agent can learn behaviors to achieve its goal.\"\n",
    "\n",
    "*- OpenAI Spinning Up*\n",
    "\n",
    "<br><br>\n",
    "In other words, given a **state (input)** and an **action space [a set of all possible actions] (output)**, to *train* an RL model means to determine the **mapping** between the state and the action space. This mapping is called a **policy.**\n",
    "- **The goal of RL is to design and determine policies**, that will solve problems\n",
    "    - by maximizing some cumulative reward over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiYAAADlCAYAAABnLTx1AAABUWlDQ1BJQ0MgUHJvZmlsZQAAGJV1kE1LAgEQhh9NKEpIpDqEkIQHDxZlH57NyAIDsaKP27qaCmrLuiH9g85Bt35BEBRdCzsUdOtQ9AuC6FiQHSy2Wa3UooGXeXh5ZxgG7CialncAhaKhJ6Iz3rX1DW/nEzaGcOPCqaglLRyPxyTCd2+v6r2kpW5HrF2P5UX36fSb5/ro6uQh5PH+zbdVdypdUqW/i/yqphtg8wnHy4ZmsYg+XY4S3rU40+ADi5MNPq5nlhMR4Uthl5pVUsJ3woFki59p4UJ+W/26wbremS6uLFld5CFGFC9zzDP7T26ynouwhcYOOjkyZDFkKiyORp608AJFVEYJCAcZE01Z//39t6aXDcG47LetNr3cIVQGoX+g6fleoXcPzi40RVd+vmmrOkqbE8EGu16gK2aazxXo8UNt2DRrdtP82IeOGzhPfQJ7El+/zetrpgAAADhlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAAqACAAQAAAABAAACJqADAAQAAAABAAAA5QAAAAAMW/vpAABAAElEQVR4Ae2dB2AURRfHHy0koSWU0HtL6L33XqVKVymigKJSlWKjieKngjQREBBQmrTQW2ihSO8t9NBLgCSUBPjyf7gxIJBLckn27v6jl9vbm51585vl5u2bN28ShIaEPhUmEiABEiABEiABEjABgYQmkIEikAAJkAAJkAAJkIASoGLCG4EESIAESIAESMA0BKiYmKYrKAgJkAAJkAAJkAAVE94DJEACJEACJEACpiFAxcQ0XUFBSIAESIAESIAEqJjwHiABEiABEiABEjANASompukKCkICJEACJEACJEDFhPcACZAACZAACZCAaQhQMTFNV1AQEiABEiABEiABKia8B0iABEiABEiABExDgIqJabqCgpAACZAACZAACVAx4T1AAiRAAiRAAiRgGgJUTEzTFRSEBEiABEiABEiAignvARIgARIgARIgAdMQSGwaSSgICdghgYcPH8revXvl3Llz+rp67ao8efLEDltq3iYlSJBA3N3dJVu2bJItazYpXry4uLm5mVdgSkYCDk4gQWhI6FMHZ8Dmk0CsEDh16pRMnTpVgoKCxCO9h6RNk1ZSpkopGCiZ4pBA2C/c/fv3JSAgQKAY3rp5S7p06SJly5aNQyFYFQmQgKUEqJhYSor5SCAKBNavXy+zZs+SAl4FpGbNmpI2bVp9Sk+WLBkVkyhwtEbWp0+fCixXd+/eVeVk2/ZtsnTJUqlVu5Z0e7+bNapgGSRAAlYkwKkcK8JkUSQAAn6n/WTe/HlSp3YdqVOnjioktJLE370B9s7Ozvry8PCQrFmzSrmy5eSjjz6SwoUKS8WKFeNPONZMAiTwHwJ0fv0PEp4ggegTwJP57Nmz1ZcBlhL4NlApiT7P2LjSxcVF8ufPLx9//LF8/fXXcvPmzdiohmWSAAlEkwAVk2iC42Uk8DIChw8flrt37krVqlV1+uZleXjOHASqVKkiBQsWlPUb1ptDIEpBAiSgBKiY8EYgASsSuOh/UaduMmXKZMVSWVRsEHBycpJixYoJlEkmEiAB8xCgYmKevqAkdkDg8uXL6suQLl06O2iNfTchceLEkiNnDjl18pR9N5StIwEbI0DFxMY6jOKam8CdgDuCAQ+rb5jMTSBhwoS6hPva9WvmFpTSkYCDEaBi4mAdzuaSAAmQAAmQgJkJUDExc+9QNhIgARIgARJwMAJUTBysw9lcEiABEiABEjAzASomZu4dykYCJEACJEACDkaAiomDdTibSwIkQAIkQAJmJkDFxMy9Q9lIgARIgARIwMEIUDFxsA5nc0mABEiABEjAzASomJi5dygbCViZQGhoqMybN09SuaWSdzq+Y+XSWRwJkAAJxJwAFZOYM2QJJGAzBB49eiSrVq2SokWLyqlTp+TvXX/Hqezbd2wXF1cX2bVrV5zWy8pIgARsh0Bi2xGVkpKAuQg8efJEED3UllJAQIBs2rxJvvv2O/lt2m+ydctWKVmi5HPtuHjxosCyYiTsjpw8eXJJkyaNnr9165YEBwfr14kSJZIsWbLoDsrgge+MFBgYqIepU6eWFClSyJ07d+TK5St67tKlS3L+/HndfRnfMZEACZCAQYCKiUGC7yQQRQInTpyQp0+fCjbsS5kypQ7OUSwizrP/OedPKVqkqGBn3Z1/7xRYMFpfbS0ZM2ZUWU6ePCmt27TW/X7QNrTRzc1NenTvIR988IFaOib9OkmQD0pZUFCQTJwwUcqVKye3b9+Wfv376bnMmTJr+VevXpUOHTrIRz0/kkWLFsn4CeNVufniiy8kS9Ys0qVzF2nWrFmcc2CFJEAC5iVgW4975uVIyRyQwMKFC6Vdu3Yyfvx42bZtm1y/fl0VFbOigJVj/vz5Uqt2LbWAFPAqIP4X/WX//v0q8sOHD6VXr15St25d2eizUTas3yCNGzeW6tWqS58+feTQoUPy89ifpUrlKuKzwUetLS1btJQ2bduoUoJCUAemiGrWrKnfDxs6TJYtWyYHDx6Ujh07ytixY3UvocmTJ4v3Um8qJWa9WSgXCcQjAVpM4hE+qzY3gZCQEMF0xL179+Re4D0JvBcoj0IeycMHDwXf+V/yl4OHDsrRY0clb9680rBhQzl/7ryULVvWlA3buXOnHD16VOrVrSdOTk5SqFAhSZkqpfqZVK1aVdsUFBwkHh4eav2BRcQjnYf4nfbT9hw+cljOnTsnF/0vyqxZs/QcOFy7dk2OHTsm+fLlE6ckTlKubDlp1KiRfo86XFxc9DpYVcyYYNWZO3euJEiYQJI6JZXESZ79LLq6uAqmqpBcXV11Y0ZMO2FaC5s0YrNGJhIgAesT4L8s6zNliTZE4PHjx/q0f+XKFblx44Ye3717V/0hoJQ8DfsPvhNPnzyVx08eh/tSJJAEcvHCRW0p/DHgM3H82PFw3wszIpg2bZoqH3/88YeKB58PWHn27NkjZ8+dFS9PL0mXLp3MmTNHgoOC1fpz4MAB+fjjjzX/g/sPlJG/v/9zPii9e/eWzJkzv7TJGNChBIElOJoxwVJ0+vRpefDwgU5hQUYon8mSJ9N+hxXI2cVZRYeigr7H/0mTJpVUKVPpCie8g1369OnVF8eM7aRMJGArBKiY2EpPUc4YE4DPBJQOOF2eO39OLly4oMePHj4Sp6RO4pzUWZIkSaJPwngadnd3V4tCyhQp1YcET8vOzs46IGFQWrFihaxctVJKlSolLZq3kFq1asniJYtjLGdsFIA2YzXO+++/r0/7qANP/eXLl5cd23fI0SNHJbV7aoH1oEGDBtpeDL7IX79+fRUJbc+bJ6+82+Xdl1qFbt68GanoYGm2lC1bNunbt6/cv39flTFYgaCs4H7BMc6HhIbI3Tt31Xp2995dtZ4hD5x9YTEKfRyqyszDRw9VccmcJbNkz5ZdHYOzZssqbqncbMIHyWx9Q3kckwAVE8fsd4dpNawge/fulcOHD+tTMQYVDI6pUqWS1GlSS768+SRN2jSSNk1adfI0zPR40scUxOtS8eLFZfDgwVK3Tl0pUKCADvRQVsyYlq9YLkmcksjAAQOfe6I/cPCAOrIeOXpEvLy8dOCFBSV5suTajL1J96oVqH379pIzZ061IK1bt06nrrDaBsrIqO9HychvRlrUbNdkrjrgnzlzRhU6iy6Kg0xQRKO6OghKC5x/jRdWPN0OuC23b92WO3fvqF/N5s2bJeBOgE4R5ciZQwoWKKhTaLCs2NqKrjjoBlZBAkqAiglvBLsiALP78ePHZceOHbqC5MbNG5I1S1bJlTuXVKpUSTJkyKBKCQYh+FdASYGVBEtio5q8CnhJnjx5VKGJ6rVxnX/69OnSrGmz55QSyJAndx6BEyziijRp0kStQcWKFpNKlStJwgQJdepmypQpaj2qXKWydHu/my4zhvUlqXNSefDggQz5eojFzcmQPoO8/fbb8tXXX+kqnW7du0nlSpUtvt5MGXHfYMUSXhETpgcxdQXl5N7dMP+kMB8lWFZgoVu/fr1MmTpFlVgs0y5RooQUKVJEfVgilsFjEnBkAlRMHLn37aDtMLdj+gGxOTZv2iyHDh/SQbRI4SI6BQErQIqUKSRF8hRqAcEUjLVSMtdkYV6R1iotdsuZ9ts0gYXjxQTLUL9+/dQqsmXrFnny+ImuNIIjK9LVa1dlydIlAssKpnjq1asnxYoV04HXKAvWIiQM0CNGjAh3GMU5xDj5ddKvOjUE9rASDB82XPsMA7uxTBl57SXBDwUWObyMBD8kKM2BQYE6DQSLEax4EyZMUP8eWFLKVygvFStUlBw5ctCaYoDju0MSoGLikN1u+43G6pGVK1fKooWLBNMQiGRaqWIlwRN4xgwZ9ckfTpcY/JhE8ufP/0oMmFZA2rN3j8DCBIdYrDIy4pjs3r1brS3IAz8TDJwvSxiQs2bN+txX6ANMAUVMcBLFy5ESpooQ6wYvCQsZkytXLo0lAz8V+D2B8caNG1WJgwLZvEVzXT0FTtGx5jkSW7bV/ghQMbG/PrXbFsFEjh/vmbNmire3t1SuXFmDd42vOT7K/gF2CykGDYMDL57kGzVupNMPKApKy4zpMzQuSQyK5qUvEICighf8mGBpggMuAs3BX8XX11dm/zFbhgwZImXLlNWpLzhWG0uXXyiKH0nA7ggkCA0JfWp3rWKD7IKAsSICsTdWrAxbAbNipTqqNm7UWFq3bq1TNmZr6HfffafWGmOJrdnkozzPE8B0Sq/evWT1qtXPf2GCT4iki+B0CIqHVWR16tTR6TRE7jWWYZtATIpAAlYnQIuJ1ZGywJgQwPQBnhqvXL0i+/ft1yW5cByE4+qMGTMkd+7cfHKMCWBeazMEsFwd4fzxwv5FsBJ+//33GuAN/j5lypQROBPDkZsrfGymWymoBQSomFgAiVlin4A6sYY5Wp49c1YQ1GvX7me7zyIkOqKSvrjyIfYlYg0kYB4CcCLu1q2bdOrUSTZs2CDLli9Tpb1M6TJSvERxyZ0rt0670afKPH1GSaJPgIpJ9NnxSisQwGoFRN2EAysipyICKUKid+7UWRAnxJqraKwgLosggXglgH8PWBlVvXp13bsIU5zYHiC9R3opXLiwxtOBozMVlHjtJlYeQwJUTGIIkJdHjwAUkpOnTso232266RviYRQsVFBatGihKxZomo4eV17lGASgoJQsWVKV99NnTsumjZvEd5uvYMm3Z35PqVixogZyo8OsY9wP9tZKKib21qMmbw8UEmwKh0BTsJRg07TSpUtLlapVJE3qNCaXnuKRgLkIQIFHkDy8sN8TdrmGgoLNJeGPVbtWbSoo5uoySmMBASomFkBilpgTwAZu2JkWc+Mnjp/QOBZVqlSR0qVK69QNLSQxZ8wSHJsAohpjyXGFChU0ku/27dtl7Lix6n8CXy1PT086jjv2LWIzradiYjNdZbuCYj+VOXPnyM6dO/VHsmGjhoLIrPAlsTdTM3akxc602Pgtsr12bLdH7UNyKMtYkqtBz+yjSdoKxJ7Bqh1M9Rw8eFCnd0aOHKlTpa3ebKUB7xi0zY463A6bQsXEDjvVTE3y8fHRsNtZsmaRTh076VNbmjRpNLiUmeS0lix4akXcFWwe+GIUVGvVwXKsQwDTinC2xn5H9pageOBeRORYREWGc/natWvl888/lzeavCENGzTUZcf21m62xz4IUDGxj340XSsuX74sw4YP00EawcawURv2DrE3C8mL4LNkziK7/t6le8FQMXmRjrk+I4AflEgvTy9zCWZFafDvDZZJPAxgc0Zs1vjr5F9lzeo10rdvX8GeSJxGtSJwFmUVAgmtUgoLIYF/CODHftr0aVK9RnVdwvjXgr+kUcNGuoGcvSslQIAN7R4/eaybCmIPFCbzEjhy5Ihs2rRJateubV4hrSQZ/u0hFlCNGjVk8q+TNThb+w7t5aeffhJYjphIwEwEqJiYqTdsWBb8uGEjsrZt28qSxUtk3rx58sUXX+iPoSMoJEbXIQrn22+9LQcPHJR9+/bxR98AY6J3+JYEBgbKwEEDpV/ffpIpUyYTSRe7osA6gnv0vffek8WLFsvefXulYqWKsmXLFsFDBRMJmIFAorDB4yszCEIZbJMANtbDbrSzZs+SyVMmC7z/Bw8eLFmzPL/LrG22LnpSw3T+8NFDmTx5smAQhDMi5vwxKNBsHj2mMb0K9yl28oVCsmfPHun5UU8pV66cdHu/W0yLttnr4fTbvFlzyZY1mwwfMVw3cIS/TbJkyXif2myv2ofg3MTPPvoxXlqBlSfw+l+wYIEEBQfJe13f05gJHHyfdQcGwFHfj9IpLUzx4MkczohMcU8g4E6AXLp0Sfz8/GT3rt3SvXt3eeONN+JeEJPWeP3Gdfn555/lwvkL0r59e40tBJ8wJhKIDwJUTOKDuo3XiX1tYCXZ4LNBsOqmYIGC0qpVK3Wys/GmWV38gIAA8dnoo06W8GkIvBdo9TpYYOQEECk1V65ckt8zv9SsUVOw9wzT8wSCg4M1ztCqVaukVKlSUrdOXcmWLZvdO6w/T4GfzECAiokZesHGZNi7d68sX75crl2/pssOsW8H9+awsU6kuCTwEgLwFYMVdP6C+fptndp11Hri6ur6ktw8RQKxQ4CKSexwtctS4Ry3dOlStZRgKWyL5i007LVdNpaNIgEHJoBpr6XeSwVWvpIlSkqTJk10ub8DI2HT45AAFZM4hG3LVd27d0+mTp2qK00Q9rpWrVrCpyhb7lHKTgKvJxB8P1h8fX1l4cKFOvXVpXMXTte+Hhm/tRIBKiZWAmnPxeDpCXtuPHr0SNq0bqOhrhnS2p57nG0jgWcEsJoJUzuzZ88WxOXBijv65/DuiG0CVExim7CNl3/48GEZO3ashldv3aa17nVj402i+CRAAlEgAGf3ixcvakiADRs2yDcjvpESJUpEoQRmJYGoEaBiEjVeDpUbZtxJkyZJ2bJlpXnz5mrGpaXEoW4BNpYEwgncuXNHvJd565Ruz549pWmTpuHf8YAErEmAe+VYk6adlIUnpJWrVsr0adNVIWnYsKH6k1ApsZMOZjNIIBoEENcEuxNjKmfo0KFy985defvtt6NREi8hgdcTYEj61/NxuG+hlCDuxrhx46RBwwaqmCASJJUSh7sV2GAS+A8BhAWoUrmK/PjDjzJv/jz1PflPJp4ggRgSoGISQ4D2dDliGGzZukXG/jxWmrzRRNq2aSuJE9OoZk99zLaQQEwJ4CGlYMGCMmjgIJ3awb5YcIxnIgFrEaBiYi2SNl4OlJIdO3ZoWGosBe7YsSODptl4n1J8EogtAth2onTp0tK1a1dBpFhvb2/BFhVMJGANAnwctgZFOyhj+47tMmbMGKlapap06dKFSokd9CmbQAKxSQC7hmNaB9YS7FQMZaVOnTqMbxSb0B2kbComDtLRr2sm4hT88MMPUrNmTen6bldxcnJ6XXZ+RwIkQAJKAMoJ9h56HPpYI8XCBwW/I87OziREAtEmQMUk2ujs48Jz586ph321qtV0d2DueWMf/cpWkEBcEYAfGiwl2ARw0eJFkiZNGp3mgdLCRALRIUDFJDrU7OSamzdvyrfffSs5c+WUd999l9M3dtKvbAYJxDUBKCdYxYddx7F1hYeHh+7mHNdysD77IEDnV/voxyi34uHDhzJ+wnh58viJ9O3Tl/PCUSbIC0iABCIScHVxldatW0vGTBnlp9E/SVBQUMSveUwCFhOgYmIxKvvJ+OTJE40/cOLECfno448kXbp09tM4toQESCDeCKROnVq6d+suQYFBMvLbkYK9dphIIKoEqJhElZgd5F+/fr2sWbtGHV3z58tvBy1iE0iABMxCIEOGDLrZH8IPTJs2zSxiUQ4bIkDFxIY6yxqi+vv7y5y5c6Rxo8ZSpkwZoYOaNaiyDBIggYgEcubMKf/73//k18m/CjYCZSKBqBCgYhIVWjaeF1M4c+bMkUyZMkn16tW5pM/G+5Pik4CZCRQqWEindb786ktGhjVzR5lQNiomJuyU2BAJe+Bs9d0qJ0+elNq1agvMrUwkQAIkEFsEELoeu5KnSplKZs2aJfgNYiIBSwhQMbGEkh3kwTK+lStXSoGCBaRUqVJ20CI2gQRIwOwEsAEoQhEsWbpE4GzPRAKWEKBiYgklG8+DkNEbNmyQ+8H3pV7depzCsfH+pPgkYCsEEKYeG/5Vr1Zdfpv2m9y9e9dWRKec8UiAikk8wo9q1fAR8dnoI6f8TkXpUj8/P53GqVCxguTJkydK1zIzCZAACcSEQMqUKaVu3boaGRYrAs06pXP79m1ZtmyZXL16NSbN5bVWIGD1yK+BgYHi6+srFy9eVPHy5s0rJUqUEJj0zJIgI7bqjvgPBOvvsUoFjqFmTSEhIfLxxx9Lh/YdpF+/fhaJ+eDBA9m/f78kCPsPTy2Y92UiARIggbgkkD17dp1C3rZtm1SqVEnSpk1rleqhRECZQPnlypWL0Thz/PhxebPVmzJ3zlxp1KiRVeRjIdEjkOiLL774KnqX/vcqbHuNVR8zfp8hTkmd5HbAbcEGcW5ubpI5U2YJDAqU8ePHi7u7u8U3JrTYX375RTzSe+h1/6016meuXLkidevVFYRRTpgooVz0vyg+Pj5y+fJlgSKVIkWKqBcaB1cgWBGW3xUpUkQqVqxoUY1o0/z586VChQq6f4VFFzETCZAACViRAH5rkfbu2yvJXJNJ7ty5Y1w6pqhXrFwhvXr30gfhIoWLSMaMGS0qF0uYN/hsELdUbgKLDhIeVF1cXHSHdWspThYJw0z/IWBViwnmD2fNniVly5SVYUOHaWU7d+4MH+jv3rkrw4YNkyxZsoinp+d/hHnZCeznMnzEcClQoIDkypnrZVmifa5d23bStGlTNTFOnz5dfp/5u5QuU9rUVpOoNDY0NFQOHT6kCiKWBzORAAmQQHwRwDRy9mzZZe/evVK2bFlJlSpVjES5c+eObPPdJo0aNpIbN2/Irl271DpvSaGwIs+cNVOyZM6i4xGuyZo1q3z91deWXM48sUzAqooJBsIX5+cwPYIEEx4sH8H3g+XnsT/L8uXLpVr1avLO2+8IlBds/IQpFqS33npL42zAijFu/DjdcwGbzf218C9p3LixNGzQULAr7vQZ0+XE8Wee3l4FvGTQwEF6fVT/uLq6StZsWQXy37t7L/xylO+zwUcwhZI0aVLp2bOnFCtWTBDRcOWqlQLFBhYWpNGjR6uCgykWPB1ASfv9998lWfJk0vGdjrJlyxaZO2+u3Lp5S/OjjbVq1RI4hx07dkygGMGqMXfuXClYqKC0bdNWnJycZOzYsXL+/Hm95r333tN3S/9gr4q1a9dK5UqVGXbeUmjMRwIkECsEMJ1fsmRJWbhwof7mQTmJboJ1A2MALB9hVn/d1Xj3nt06/qRPnz68WIwtY8aM0c+wgrz55pty/cZ13Sfs7NmzMmToEEmXNp0MHDhQrfj9+vfT6fISxUvoNVBgMANw9cozvxPI37lzZ1Wq4PP322+/CfYdgyuAt7e3XoOHXUwFOTs7h8vBg6gRsKrza/LkydWPAZpo/0/7y5kzZ8KlyZUrlzRp0kQHWzy9v/POO1K+XHn1f5i/YL4O0uhwWEa+G/WdmuZw3LJlS70GzlPt27WXwoUKy7Hjx9SK4uLsojcJrlu5YqXeXOEVRuHg2rVrumoFU06IWIj0448/ysaNG6VFixZah5eXl7Zp3759qsBAOdm+Y7vmxXTToUOHVImCcoMEbX7xksVqHkS+jZs2SuXKlbWsYsWLyddDvg5XOK5dvyYTJk5QRatevXpSv1593ekXHM5fOK/znmgjLEfXrl7T8iP7g3+4UHiwRK9RY86XRsaL35MACcQ+gcJFCotrMle1mgQHB0e7Qjwsbtq0SVKkTKFT1Bgr8FuH3zwj4Xf3gw8/kHLly+nvLnwd//rrL8mZI6dUrVpVLSUNGzbU7zAFBHnge3jJ/5IWAcvO4MGDJWGChJoHv8GYOho0aJDAbQG/sYgNhQ0Ljxw5onkwRTVl6hTZvXu3IQbfo0HAqhYT+GZAe83vmV87b8mSJdKqVSvp0b2HBvQqXbq0JE6UWBARsFq1aiounuo/+ugjSe+RXgdjbCi34K8Fqg3j5oHykiRJEilWtJgO7LhowoQJEngvUHBT4YZEOn36tPTt11etGpbOM/bu01u++vortYgUL1ZcJk2aJBkzZFQlA9YZ3JS1atbSsO1Y8rZ8xXJZ6r1UnU8h57Gjx9Sag6Bl+fLlk3Pnz8mBAwf0qQAKDKahoHlDKcuRI4ekdk+tShaUn3Fjx2kbcR4J85z169dXXrCUbN26VV9ffvmlKioIHY+txMHTkoR/NPCAB0OPdB6WXMI8JEACJBCrBFIkT6GLDP7++2/16Yuurwn8S7yXeUub1m3ULwRjCn4j9+zZo+XDVwQPlyi/c6fOuns6lIkaNWroWFTwcEFVjkqWKBnur4eHSSPhGOUnTpJYunfvrr/h+A7xoAYNHiQ9evSQ/Pnz62IC/J5/8MEHAksN/Cd37NwhBw8dVGdfWNqZok7AqhYTTEtgwO76blc5eeKkapCTJ0/WqZsbN268VDqY97Zs3iLZc2SXTJkzSc1aNdU8d/nKZYGp7MUERQbOqgjYU71Gdb0G13362ac6FfQo5NGLl7zy85dffCnLvJepMgOT3aGDh3TK5tKlSwIrSIcOHXSKB+XDygGlAwpRtmzZpHjx4nL23Fm9UeHgi/lTWHWwIglOqvgHgn8UhQoVUh8b76XeUrhwYZUXGvyly5fCVy5BQHCAcgOlBAm+IWnSpNE5WWM/G0wbJXFKot9H9gfsoNzUrFGTK3Eig8XvSYAE4oQAVgXidxBT2i9O+0dFgJ1/7xRMxeBhFwlKAvwWcR4O/0j4TccDJabqkaCswL/RcMTVk6/4A5cDTBVhPDOs6Mjq6eWp45Kx6hTtSZ4suSol+B7TRfCdgQKDFZFM0SNgVcXEEAFaIp7u+/frr9rm5s2bw28WIw/eMTc39bepMmz4MJk4YaKa0DZv2izwrsbAiqf+FxPOPX3yVKdYdu7YqdfA9Hbzxk0JDQnVgfzFa171GTdQ5syZpUH9BlK0WFFZtGiRBAQEPFOIwqrG9BDKxuvypcty4/oN+fbbb1Uzh4UFsmBaCXOLKKdqlaoyecpkneqBstKsaTPNO3HiRBn1/SjV4FHW3j17Nf/jJ/9uCY4bHCZDI0G5gaKHV3QSVh6dPnNaV/BE53peQwIkQAKxQQAOsHjYwgMgpmSik/7840+dCsdiBVg10nmkE/zO+vj4qPUcZeI3NFHCRNEpPmyJjug4gN9lvIxk/Ea/7KEZeYz8GKOYok8geqPeK+qDogFLg5Fw08HkhqXCmFeEtorlucbNiPeA2wEC/43y5cvrZZjnCwn992aFtgoFwDgHPxYoPZgmuXXrVrjyomWFKRVIuGnu3bunfh6vuoE04z9/YAFp2qSpmvZg8YBJDhYMKB2QHwkywDnX+AwP7iePn8jq1avVygJtHKHeoa1DU8eyOCg7aA/khGNr0aJFtSyUE5lcGdJn0PrAE3Ujoc3nz53X48j+QC748hjWlsjy83sSIAESiCsC8DWBD2LE6RNL68Y4M/uP2fLb1N/0YRQPpHgd2H9Af2MxlYLFB7BAY5rdGG/g/4fz+O11c3fTZcuv+h2G42qmjJn0txsPeUjICysNlA88iDLFHgGrKSboNEQlbdO2jfpowBkUq1B279qtfg7wIUFCh8JXA85CuEkwwMMSgVU6uAaORC/uqYApjbVr1goC4MCigUA6GKyxvHf79u16HfxZBg4aqHVgAO/bt69OJSGWiiUJN3HadGkFc58w/5UpW0aXPENRgVyYmhn57UiB7wgS/EYKFS6kigkcsFK5PVv6BsfVTz/9VJetFfAqoGbEHGF+JPhHuG7dOi1rxIgR8qqpLUNWrGaCIgcLjO+2ZzK079BeFTwjz+veV6xYof4xr8vD70iABEggPgjgIQ1T8hgDopqw2hG+HLVr137uUvxeZsuaTafkMU7AtwRjEsYh/IYjENvnn3+udcIf0v+Sv2AlD74zHjiNAo3yL1y4IDNmzNA8m7dslh9/+lE3JjRWYxr5+W5dAlYLsAYtEmvCkzolFSztXbV6lVoO2rdvL3gZ83zwuYBighUvmEqpWbOmwC8EDq+4Bp8xrwflA3FLMJWRI2cOdUSCkoPv4NAJZ6cd23foEmJchzm/KZOnKB1YS7Cc19nFWZcWv+iABI0bCgacTWEtQYKGjJsTlhk477Z6s5WujZ8zd45ufgc/GKzQqVa1mubHH5QDC0mlipWkVMlSajmBwnTk6BHB0l7j5oUyBsvJwkULtY3NmjfT6agqVaqoHwr+cUL+GtVrqDUIZcMZFnOmWP0DpQtt/OGHH5QvlCgoT69KcPKCkjbk6yECCxMTCZAACZiJABQDKAyYEsdiBYwfliZMuZcuVVoXP0S8Br/zmL7BbymWIiPCLBSVKVOm6O8nrM1YDYrfT0wn4TcZygp+W6Hk4HpYuzEuQMnBAyV+w7EQYunSpQKXBISr+OTjT9Sijt96v1N+an2pVq2aioIxAcoMrsVvtOEzGFFOHkdOIEGYCcyuJsNguUGgncGfD5b333tflYnIMdhXDlijWrVupU8O9tUytoYESMBeCPTp00cXO9SuVVtXXtpLu9iOmBOw2lROzEWxTgnQWKHZeub31FUy1inVtkrBE0NET3Lbkp7SkgAJOAIBrGBBDClYOZhIICIBq8YxiVhwfB3D2RPzlw0aNnDYaQw4a8Fxi4kESIAEzEoA0/JYNmwEpTSrnJQr7gnYnWKCOT2EenfkhEiyGTJmcGQEbDsJkIDJCWCxAbbboMXE5B0VD+LZ3VROPDA0XZXY1wGOyEwkQAIkYFYCaVKnkZu3blIxMWsHxaNcVEziEX5sVY39dbAMm4kESIAEzEoAKw+xgvJVsUTMKjflin0CVExin3Gc1+Dq4ipBwUFxXi8rJAESIAFLCSAUBJQSI4Ckpdcxn/0ToGJih32cPkP6l24BYIdNZZNIgARslAACYSJ4piV719hoEyl2NAlQMYkmODNfhpD6RhhlM8tJ2UiABByXAMLRu7u56zYljkuBLX8ZASomL6Ni4+fg+IoNsphIgARIwKwEAu4EaPTvaG+0Z9aGUa4YE7C75cIIsAbnT6yPR5hjbMaH8MIIfx+VhLDuZ8+dleCgYA1LnCRJEl3aFhgUqJsSZshg3uW4mbNk1s2motJe5iUBEiCBuCSAPdLwuxxfG43C8RYPcNgSJKlzUo39BCtO7ty54xID63oJAbtTTNZvWC8zps+Qu/fu6u68UE6woV+jho1e0vyXn8KeOdijZvGSxfLnn3/KyG9GCjbq275tu+7B0+uTXtKpU6eXX2yCs7CY+Pv7a+Aizt+aoEMoAgmQwH8IXL5yWZWA+FBMsInqggULZIPPBnnw4IG4OLtIlqxZdJPYXyf9+h9ZeSJuCdjdVM53330nsBjMnTNX/vf9/yR//vxq9YgKVlhdoJz07dNXN2HCLsYXzl+Qjz/+WF/YYNDMKWOmjPoU4ufnZ2YxKRsJkICDEoC14urVq7qBX1w/PMEajo0A5y+Yr5v3zfx9pgwcOFAmTZpEa4lJ7ke7s5jgptNdgkNCxMvLSzp17CRubm5Rwo2dL+vUqSO3bt3SpWzYiRK7Bbu4uEiXzl2iVFZ8ZE6SOIn+g1u7dq0qZvEhA+skARIggVcROHr0qKR2Ty3pPNLpDvKvyhcb58+ePas7xpctU1aaN2uuW5cgPD6s6zjHFP8E7M5i0rlTZ/Hd6is/jf5J5w9h3fD09FTS+MewePFiuf/gvkXkDx06pEpOt27dVCkxLsIytzVr1sjRY0eNU6Z7x9TVuvXrTCcXBSIBEiCB/Qf2S/Yc2SVVyqj5/lmD3PHjxwXTSCVLlhR3d3ctcv/+/WodL1KkSHgVFy5cEG9vb4ZeCCcSdwd2p5i0bNlSPuz5oRw9clRGfDNCcMMZafWa1bLz750SGhJqnHrt++HDhwVLbwsUKPBcvjNnzsjyFcvl+rXrz50304fSpUvLlctX1FxqJrkoCwmQgGMTCAmzZh87ekxy5sgpiP4a1wn+JUmdwpxdM/270enCRQt1GgdxVZCwfw8eTOFnyGCVcd1DInalmHz44YeSOnVqadWqlXR5t4t6Wx84cECp/v777zJlyhSZM2eONG3WVN8jww0lpmDBgpI0adLwrKdOnZKhw4aq41S//v1kyNAhplyai+korwJesn379nDZeUACJEAC8U3A2LgvY8aMgtWOcZ0QbTaJU5Lw33Xfbb76O1mpUiUVBbsdb926VUaNGiXLly2Xzp07y8SJEwUrdpjihoDd+JjA0XP6jOnSokULwQ325PGT55xeS5cprUuHe/XqJSVLlFSnKyDGdZ/0+kRX7lStUjWcelBQkBw7dkwa1G+gc4/GF9gRE9dDWenfr794eHhI2rRpja9N845wz7Vr15YlS5dIkyZNTCMXBSEBEnBsAnv37dVlwohQHR8pWfJkumLz6rWrsm/fPtmxfYcEBQaJWyo32b17txw8dFBqVK8hJUqUEEztvPvuuxqhFqEnmOKGgN0oJokSJ5I6tetI5y6dBaZCLJlt3769NGr0bJlw4L1AjWcCnxMvT69wuliBc+LECbl39174ORxcvHhRN5iCAy0GeSNhGihhooRSqmQpnaOEw5QZE+SqXKmyzJo1S/+xYT6ViQRIgATikwBihuzatUv9/uJrB/RyZcvJ+nXrpUf3HtKgYQP55ONP5JTfKfnhxx8kQcIEumACCyiwh0/lypWlUKFC8YnMIeu2G8UkR/YcOr3yql40/jFgq20jQYGBE1aunLmkTJkyxml9xzLjQwcPPXcOH65fvy6XL12W5s2bP2dJ+U9GE5yAY1ftWrWVCxUTE3QIRSABByeA32H4bxQtUjRepnGAHwHUMK0fMY0bO07wMtLuPbt1cUP37t2NU3yPQwL/mgLisNL4qAoWEGjAmLqZ/cdsPUZgHQRNmzBxgk7JWCIXHKGwDw2melatWiUnT5605LJ4yePq6irVqlWTk6dOqlUoXoRgpSRAAiQQRiA4OFitt2nTpNVQDmaFAh+TS/6XdEXmtevXZP369WpBN6u89iiXwygmGKAxnzjxl4k6xYGpDjiIjh49WmBtsTRlzJBRl7lBmQm+HyzZs2e39NJ4yZc5c2YpUbyELnvDkwoTCZAACcQHgSNHj+gy3TJly4izs3N8iGBRnQj4hoUDri6uMmb0GMEDHrY1YYo7AgnCfCaexl11rCmuCcADHRr/3Hlz5dP+nzKyYVx3AOsjARIQBL6EpRr+fAM+GxDloJdE6FgEHMZi4ljd+m9r4bgLB15o/IjjAnMqEwmQAAnEFQE8HEEh2b9vv1SpXIVKSVyBt+F6qJjYcOdZKjriBcC7HMufMZ0FXxsmEiABEogLAtgXZ926deKe2l2qVv03JENc1M06bJMAFRPb7LcoSQ2rCWKv5M2TV7D7Mpx3mUiABEggtgnArw0rXBBFtXWr1rovTWzXyfJtnwAVE9vvQ4tagNDP2JgQ+/xs2bpFEL+FiQRIgARik8Dt27dl9qzZUrNWzfA9y2KzPpZtHwSomNhHP1rUCqzfhyl1y+Ytcvr0aYuuYSYSIAESiC6BefPmaZTsxo0aPxeoMrrl8TrHIEDFxDH6WVuZKFEiqVWzlri5u+kuy3iaYSIBEiCB2CDg6+srf875U7CHWXxs1hcbbWKZcUOAiknccDZNLcmTJ5d3u7yr+0F4L/PWIEKmEY6CkAAJ2AWBc+fOybtd35W+ffoKomgzkUBUCDCOSVRo2UlerMpBxNqu73WV4cOG66aHdtI0NoMESCCeCSBmSZOmTaRZ02by/vvvcwonnvvDFqunYmKLvWYlmeEEiw2sFixYYPoItlZqMoshARKIRQJwqu/Xr5+EPg7VvWfMuslpLCJg0VYgwKkcK0C01SIqVawkPXr00G29sTkhEwmQAAlElwCUkqlTp8q1a9dkyNdDTL/JaXTbyetinwAVk9hnbOoaOnfuLBUrVZTPv/hcLl++bGpZKRwJkIA5CUApWb16tWzavEn69+8vqVOnNqeglMomCFAxsYluil0hsXcFnGJHjxktcFpjIgESIAFLCWCX9s2bNwuc6Tu07yAFCxakX4ml8JjvpQSomLwUi2OdTJo0qQwcMFCSOiWVXyf/Kn5+fo4FgK0lARKIFgFYSrZu3SpLly6VunXqSvXq1TVuSbQK40Uk8A8BKia8FZSAu7u7dOvWTVxcXGT6jOly8tRJkiEBEiCBVxIICQmRv//+W5YtXyYVKlSQunXriqur6yvz8wsSsJQAFRNLSdl5PnjPZ8iQQd5+6239cZn5+0w5deqUnbeazSMBEogOgdDQUN0DZ/78+VK8WHGpX7++JEuWLDpF8RoS+A8BKib/QeK4J6CcZMmSRdq1bSfOLs4ybdo0jXfC3Ygd955gy0ngRQJQSrZt2ybTfpsmxUsUl8aNGzOy64uQ+DlGBKiYxAif/V0M5SRr1qzqxIYw0qNHj5b9+/cLdgllIgESIAH4k/zvh//pvlsIoubm5kYoJGBVAgywZlWc9lMYrCS3A27LooWLZOWqlfJRz4+kfPnygv12mEiABByTwLhx42TR4kUyYMAAKVe2HH1KHPM2iPVWUzGJdcS2XQG87tetWyejvh8lPT/sKc2bN7ftBlF6EiCBKBNAmPnPPvtMDh48KL/88ovkypWLDylRpsgLLCWQ2NKMzOeYBLCUuEGDBpI7d25p36G9rtb5tP+njgmDrSYBBySAXcixIZ+zs7MsX75c3x0QA5schwToYxKHsG25KuwQunLFStmyeYv+SJ09e1ZCQkNsuUmUnQRI4DUEsBz40KFDgujQnvk95ddJv1IpeQ0vfmU9ApzKsR5LhygJTrDDhw+XnX/vlK5du0qVylXU+Y2bdTlE97ORDkDgyZMnEhAQIGvXrRUsB65Vq5au1EN0aCYSiAsCVEzigrKd1QFLydo1a+X3mb9LkcJFpGmzppIrZy5xcnKys5bGvDlQ5K5evSp3796VO3fvSHBQsHD5dcy5RqUEKM1JnZNKyhQpdVlr+vTpGZ30FQAfPXokx08cl6VLlsq58+ekXbt2UqF8BUmSJMkrruBpErA+ASom1mfqECXiqerkyZMyffp0CQwMlLr16krFChW5dDBC79+8eVO2+m6VAwcOyL279+TOnTsSFBRExSQCozg5TCAa0RjLWlMkTyHZs2eXGjVq6LL4OKnfBiqBsgzledOmTbJu/TpJkyaNhgzImTOnDUhPEe2NABUTe+vROG7PjRs3ZKn3Uvl759+S3zO/NGzYUHLmyOnwHvt46vRe6i1+p/2kVMlSki9fPl3JgOi6CRPStSsub1Nd+h7mwHn+/HntDx8fH41S2rBBQ6lcuXJcimLauvCQgX/HYAQLSb169Rg0zbS9Zf+CUTGx/z6O9RZiSfHu3btlxcoVEhQYJBUrVpQ6depIihQpYr1uM1Zw7do1GT9+vCRMlFBatmgpnp6eVEZM1FFXrlyRZcuWyeo1q2XokKGqNJpIvDgV5d69e7rSxnebr6RLm04fLIoWLcr7NU57gZW9SICKyYtE+DlaBPBUetH/omzdslU39sLnNm3aSJkyZaJVnq1eBP8b7DMEiwmcg+F7Q8dg8/UmpiJ/+uknOX36tIwcOVIczbET7d+xY4f88ecf6j+CTfiqVqkqadOmNV9nUSKHI5Doiy+++MrhWs0GW50ABt9UKVMJlhXnyJFDfSnwo3fkyBF9InUU68lpv9OyevVqqVypspQsUdLhp7SsfqNZqUDcr1mzZdWoxunSpRNH8qXw9/eXCRMnqAM7phlbNG+h/mHchM9KNxeLiTEBTnbHGCELiEgA3vteXl7Svn17GfDZAHny9Im89/57Mnv2bHnw4EHErHZ5fObMGQ3TnTdvXq5kMHkPp02TVkqVKqXTkCYX1SriYcoVAdL6f9pfAu8FSvce3XXVDfyf6PdkFcQsxEoEGPnVSiBZzPMEYBrHXHX2HNnl4IGDMmnSJPlr4V/S7f1u6nCIiLL2mC5duiSuyVwFS1KZzE0ASjQUyPnz5ptb0BhKh92AsRHnxIkTdcn6O2+/o/teYZUSFZIYwuXlsUKAikmsYGWhBgG3VG7qDFu4cGFZv3697rkz6ddJ0rtXbylZqqQkSWxf8RGuX7+u8VxSpUplIOC7SQlgQ8rMmTLL+QvnTSphzMSCH4mfn5/88MMPcuDgAXn77beladOm6uRKhSRmbHl17BKgYhK7fFl6GAH8COLpDBsA1qxZUxYtWiSffvap+p5gY0CYkmFBsQcnUTj9ItlDW7Qhdv4H/WT0mT00FW2BhQRB/UaPGa3+Tm+99ZYMGTJE4EvDRAK2QICKiS30kh3JCEvCO++8o0oKtk/v26+vFC9eXNq3ay9ZsmTRJcaMMmlHHc6mxAkBRBhGoMPLly/LvHnzBP+2mjVtJitXrpSMGTLGiQyshASsRYCKibVIspwoEcAqnbc6vCVvNH5D5s2fpxaUYsWKSaWKlXRlDwKROdoSzigBZGYSCCMAh3LEzTlx4oT4+vrK9h3bpWzZsrJk8RLJnDkzGZGATRKgYmKT3WY/QsOC8m6Xd6Vtm7ayYsUK8V7mrZuHFS5UWFf35MqdS7B6Av4ATCRAAqJTTwiMBv+Rw0cOy4H9B+TylctSokQJGTNmjMbOoQ8J7xRbJkDFxJZ7z45kRwyFli1bSuPGjWXXrl26Zwd+dGGGzp07t3h6eUqO7Dl0zxM7ajabQgIWE4D/CHxHDh48qArJuXPn1H+rTOkyUq1aNfHw8KBvk8U0mdHMBKiYmLl3HFA2OMEipD0ixiIqJ8zTWOq4b/8+cXVx1eWdBQsWVGUlcWLevg54izhck2/duiVHjh6RQwcPqQ8J4pG4ubtJ0yZNBdOfXAHmcLeE3TeYv+x238W22UA4wCKKLOJM4CkR1pMTx0/Ivn37ZPOWzbokF8Gx4DibMwc3DbTNXqbUryIAR1YoI7t37ZaTp05K8mTJNT4OghcWKVJEFXN7jQX0KiY87zgEqJg4Tl/bZEsxV54xY0Z9YS+PCxcuyNmzZ/XHev++/brqAE+McJqFlQV56Y9ik13t8ELDkRVKyI7tO+TAgQMCi2CmzJn03sZ0Zvbs2XXZvcODIgC7J0DFxO672H4aCCtKrly59IVNx7BL7MWLF+Xw4cPis9FHZs6aqcuNS5YsKaVLldYnS+7/8az/sXKjY6eOurEglpEymYPAjRs3NCQ+VtMcP35cnJI4Sb78+aRWrVq6fw8U7dSpUzNCqzm6i1LEEQEqJnEEmtVYl4Czs7PkCNssEE+RmM4JCAiQ2wG3BXvVwJLy448/iv8lfylSuIiUK1dOoMhgWsjMafKUyTJ58uTn9hRCaPsePXpIkzeaxEh0PI37+PhI/Xr1Y1QOL44ZAfiHHD16VKcjt/lu0x254SdSqFAhqVOnjqT3SK8+IylTpuReSzFDzattmAAVExvuPIr+LMIqrCJ4IW6DZ35PqVmjpjx69Ehu374tW7dulW3bt8n4CeMlJCRE9+/BQIB9fAoVLCRp0qQxzUqG69euy927d8Vng09412JayhpWH7A5d/acXceGKV+hvDRq1EgGDRwUzi8mB/DzcHV1jba1AiHhcQ8eOnRI9u7bq5aRw4cOS4aMGaRC+Qry/vvvCxy54Svi5OSkigiX+cakx3itvRCgYmIvPcl2KAHMy+OFAQVh8LGdfYcOHQSRMeGbsvPvnbJnzx5Z+NdCOXrsqJrJS5YoKSVKlhBsAQ+FJT5XOUARQehwa4e0N8q159sEq1eCAoNi3ERMqQwdNlTvhQ96fKD3kiWFBgcHq3P27t27ZdfuXaqIBAUF6cqZ0qVLCzbPw33m7uZu9f61RD7mIQFbIUDFxFZ6inLGiAAGZjgQ4oVgbkiwqmBJMpYj7z+wX0aMGCGn/E7pE3KmjJkkc5bMGkclU6ZMkjVrVsE75vxhwUB5hvKAd+NJ9/6D+/odnpaNczES/J+L79y5IwMGDBBnF2e5fOmy+G7zFQyEjRs1lnHjxsm2bdukd5/eMmb0GKlSpYpehQG263tdpUuXLtpm99TuMvKbkdKzZ0/ZvHmz9OnbR8qXKy/LVyzXKa+hQ4dq+xBPBkob4makSJ5Ctw1AlF4XFxeZNWuWeHt7S548efQ6hECHQnD1ylVV6G7evCnffvetnPY7rUrhrNmz5P79+2rFaNWqlVSrXk0/w0qA6TZMUYEfBvDBgwfLH3/+ofxwzYL5C6Rq1araTwgchuB7jRo20j1g0HdvvvmmDBwwUKfxWrVupYrn2HFjZdr0aQJH6T/++MMi9Ggnyrtw8YK2D9Np2CUaWyTcvHVT5UVBsLihvfjuov9Ffcexv7+/HqMcrCQrWqSoxuQZOmSoZMuWzSIZmIkESOBfAlRM/mXBIwcjAPO5p6envlq3bh3eevirIJImFAAMOjjGlBAGLgxE8NeAcoKBOkHYfy6uLvpUjQEWygAcb+GYC0UmqinkUYia/g2lJ1nyZLoc2ihn5syZ8s0338iUKVNUqcL0BRwlMYBD6Vq3bp3GgYHiBKfglClSCgJwoa0vJkwx5M2TV1auWKnXYhqpdu3a6nyJKLxwNt6xY4d07tJZLRG9evXSIjAtkdQ5qSxetFgH7xo1a0jLN1vKmtVrwqvY4LNBZbrkf0mVjYEDB8q48eNk5u8z1edn1Pej5KuvvhJYq8ARbXoa9t++vftUOcJGj81bNJeNPht1k0cUjJUqlStXFpSJ6bnevXvL6jWrdZ+lo0eOSn7P/NKieQtVMMMFseAACgf6d/iI4fqOz0hz/pwjO3fslCROSVTJhPKEgH/oV6yWweaT1atV1ylEnIOVzug3C6plFhIggVcQoGLyCjA87bgEMAWEl5en1yshYBDHUz2ekjFgwR8Bx9OmTZMUKVMInBejk27dviV9+vQJc555djUce78d+W14UbASNKjfQAdBOExicITy0KRJEx20V69erUoRooBu2bJFihQt8spdZeE4DOsIFBokbPgWcCdAhg0dFq7IIG5GvXr11Fnzww8/1HxYGfVe1/dUKcGJ1q1aS79+/Z5z2sU1mEJDQhuwsqRv3766jwssSZUrVZapU6cKopeirbBUQYGCYyhe4Id8a9etDVdMsNqq1yfPlKOCBQpKlsxZdI8Y9AOcoaObMPUHx2jsfg0l6cjhI6qMQtmDFQdTa/E5vRfddvE6ErBVAlRMbLXnKHe8EsDA+TLlA0/OmKaI7gaEWIWzatUqi5+8UR8C0MGCg2kZXIuBHVMT165f0x1mU7mleikrDOYR2wBLEQZgWFuMhGP4RBx7cEwwnfSylDZdWlXKrl+/brE/RsRy7gffV0sUlJBjx46FfwUFDM7ML0uQG0oEZA59HPqyLBafgwKEaZse3XuoxWXDhg2yctVKVUyNqTuLC2NGEiCBGBOgYhJjhCyABMxBACtvYEHBXkOYbvJI56EbISZJnMQiATHQw/oA/xgj4Tj4frCgDHwfGwlTJbDwdOzYUVq92eo/VUAmS5I1rBpQDNu0aaNWIqyogZLJRAIkELcEEsZtdayNBEggtghgygFxW46fOK6+JojzEhXnywIFC6hSgukgrGKCUgKfGawyQVRdayxbflnb4WgMqwymnuDTY6QJEyboclvjc2TvadOmlTNnz0SWzaLvjRVdmOZhIgESiFsC/FcXt7xZGwm8lgBWtbzf7f3wPNgjpW7duuowGn7yFQd4ui9cpLCuXvHz89OAXVGZUirgVUCGDx8uo0aNEkQihSPnrZu3pGKFiuH+Iq+oOkanoTzBWjJp0iT56KOPJE3aNFpe9mzZdem3pYW3a9tO+vXvJz0+6KH+KiiTiQRIwPYIJAgNCX1qe2JTYhIwJ4HvvvtOzf8ff/xxlAU8efKk4PX4yePwaxGiHM6pcFSF7wiUBThqGs6eO3bukMSJEgscQ5HghIt8iDCK4F3u7u7hZS1bvkygfCC2CxQgrMqBlSRd2nTheXCAlT2YvkGCslOieAmBNQIJWwDAqgEZ4NCKhCW0e/fulerVq6t/yqlTp3QqqXDhwrqyBzLt3rNb8uTOoytaVOEJW2KMDRkRNwZB7rASBquIsPLJSGXLlNUpHlhvENEXYfWxcaOxwgh1JkyUUH1BcC4oOEjWr18vCRMk1JUyCKQXWUKdvXr3ktWrVkeWld+TAAnEEQEqJnEEmtU4BoGYKCaOQchcraRiYq7+oDQkAAL0MeF9QAIkQAIkQAIkYBoCVExM0xUUhARIgARIgARIgIoJ7wESIAESIAESIAHTEKBiYpquoCAkQAIkQAIkQAJUTHgPkAAJkAAJkAAJmIYAtrftoQAAE0NJREFUFRPTdAUFIQESIAESIAESoGLCe4AErEjAKamTPHn6RONyWLFYFhULBLDpYnBwcHhMmFiogkWSAAlEgwAVk2hA4yUk8CoC2J/m4YOHUQql/qqyeD52CYSGhsr5C+ejFLY/diVi6SRAAiBAxYT3AQlYkQA20gsMCpQrV65YsVQWFRsEQkJD5MSJE7rRYWyUzzJJgASiR4CKSfS48SoSeCkB7O4b8ihEQ7RbuivuSwviyVgncP7cedm+fbtUqlgp1utiBSRAApYToGJiOSvmJIFICXh4eEjNmjUFe9hgzxgmcxLA/js//PiDVK9WnRYTc3YRpXJgAlRMHLjz2XTrE0iUKJFUqVJFPPN7yogRI2Tz5s2CQZDJPAQuXLggnTp3Eji/du7cOXxTQPNISElIwLEJcBM/x+5/tj6WCNx/cF/Wr1sv34z8RsqVKyfVqlWTokWKStasWWOpRhb7OgLXb1yXw4cOy44dO+TPOX9Kl85d5J133pEUKVK87jJ+RwIkEA8EqJjEA3RW6TgErl69KjNnzRTfrb6yectmuXXrluM03kQtdXd3l2LFiknZsmWlffv24uXpZSLpKAoJkEBEAlRMItLgMQmQAAmQAAmQQLwSoI9JvOJn5SRAAiRAAiRAAhEJUDGJSIPHJEACJEACJEAC8UqAikm84mflJEACJEACJEACEQlQMYlIg8ckQAIkQAIkQALxSoCKSbziZ+UkQAIkQAIkQAIRCVAxiUiDxyRAAiRAAiRAAvFKgIpJvOJn5SRAAiRAAiRAAhEJUDGJSIPHJEACJEACJEAC8UqAikm84mflJEACJEACJEACEQlQMYlIg8ckQAIkQAIkQALxSoCKSbziZ+UkQAIkQAIkQAIRCVAxiUiDxyRAAiRAAiRAAvFKgIpJvOJn5SRAAiRAAiRAAhEJUDGJSIPHJEACJEACJEAC8UqAikm84mflJEACJEACJEACEQlQMYlIg8ckQAIkQAIkQALxSoCKSbzid4zKnz59Kk+ePHGMxr6ilWj/wYMHHZZDUFCQnDx18hV0bOv0o0eP5MiRI7Em9N27d2X+/Ply7ty5WKuDBZOAmQlQMTFz79iJbHXq1hGnpE6yZMmSOG/RvXv3ZOfOnXFe74sVjh03VoqXKC6/Tv71xa9s/rMljN96+y3x8vKSQ4cOWdzew4cPy9WrVy3OHxsZX9a2L774QooULSLe3t6xUaUyatO2jezZsydWymehJGB2AlRMzN5DNi6fj4+PbNiwQVuxcePGOG/N7zN/l44dO8Z5vS9WmCljJj2VIX2GF7+y+c+WMM6eLbukSJFCkiVPZlF7Q0NDpVLlSrJlyxaL8sdWpukzpv/n/smUOZO2JU2aNLFS7dmzZ7Xc7Nmzx0r5LJQEzE4gQWhI6FOzC0n5bJdAlapVJOB2gOTJm0eOHT0Wqybwl1Fq2qypnl60cNHLvuY5KxCIDca7d++WsuXKyo7tO6RkyZJWkDJ6RbzR5A1JmDChxOX9M+KbEQKrzOVLlyVdunTRE5xXkYANE6DFxIY7z+yir1y5Unx9fWXo0KFStWpVOXHyhFy6dOk5sY8eOyoY2NxTuwueRLt06SK9evWSWbNmaT74ZTRv0Tz8+4EDB4b7aRQpUkRWrVola9askUaNG2keXI+n7YsXL8o7Hd9RczumcjDAfDPym+fqfvHD6+r68MMPZeiwoXL8xHHp2Kmj1gWlC+1BG/LlyyczZ858rsgxP4+RAgUKyI0bN+Srr77SYyMDZF++fLn8MukXyZEzh0z9bao8fvxYfvzpRylXvpyWX616Nfn777+NS+R1MiAT5EB9p8+clkGDBmm54ILpkK1btyoDMB49ZnR4mTh4Xbutxbh2ndrSrVs3rRec6tWvJ4GBgdK/f3+VE/wgI9KEiROkW/dneft/2l/lPnP2jH4XmawvMsVFEfknTpJY77Pfpv2m5eEP7lPj/ildprQsWLBA/P39pXOXztpHuH9wj0Z8DRs+LPz6O3fuyCeffKL3AO7jtu3aPnefv45heCERDs6eOaufDKUksvsCmdFuMEX9qC/ivYh/S+AP3j179tQ8yLtt2zath39IwGwEqJiYrUfsRB44ew4YOEAKFy4sjRs3looVKmrLNm/eHN7C27dvS82aNVXR2OizUd7r+p7AdH474Lb+uGJQhl8GnlhXrlipCs53o75TRSQgIECOHD0iP4/9Wbq+11WaNWumZeF6mP9dXV3VpwGV1alTR1q0aCFVq1QNr/vFg9fVhbw7du6QZcuWSeNGjXUA+vCDD1XpmjxlsuTLm0+u37guixb/a5W5efOm9O7dW2rXri1p06aVXbt3qcKEsgzZh48YLl9//bU0bdJUihUtJh3e6iD9+vWTVq1aydifx8q1q9cE/jnXr1/HZa+VAd+fOX1GlT+Ud/nKZa0fg+4333wj7Tu0l0aNGkmJEiWkT58+Okjhmte125AzpowxsGI6L0mSJKhSrWZr166VZs2bKYvPPvtM+Q0aPEi/z58vvyRKlEiP3377be07j3QeFsn6IlMokqVKl5LFSxbLiBEjZJn3Mrl27ZpAmUCaMnWKKiXJkyeX6dOma//+tfAvvX/y5s2reerXqy9vtnxTXy1btFRl9+qVZ74vYFS6dGnt+y+//FItHWDern07vTYyhprphT9+p/30341xOrL7YuIvE1V5y5M7j0ydMlUVPSjPxhQqHHV37dolDRo2kJCQEPnxxx/lwIED0qt3L6MKvpOAuQhgKocvMrD2PTB3zlxMET5dvGix3l/BQcH6+d0u74bfb3/+8aeeO378uJ47f+68fp48ebJ+LuBV4Gnx4sWfPrj/4On94PtPv//+e/1+m++2p/v27tNj1HHa77TmX71qtZ6b8+cc/bxh/Qb9bMiANvpf9H+6ZPGS8FeYg2WkdeG6MP8ILct7qbfmv3f3nn7u3Kmzfm7btq3mefjgoX4eNHCQfo824fpcOXM9rVevnh5HlB3y4PuwqQLNHzY46mecM/jg3RIZxo4dq2W8+eab4WUYcodNiei5GdNnaJ4wJ1T9HBuMIWvEl9Gv3377rZ5v2bKlytC/X//wfBUqVHgapuCFfw6zsD0NU2rDP6M8S2U1mAYFBmmZuIfuBNzRsjZt3KR1z583/+nFCxf1uGnTpk9DHoU8VxfqW79uvX6P+8VoT9j0ip4z2tK7V2/9fOzYsfA83bt113Nod8S+ftV9apRtvHt4eDyFTPgc2X1x7uw5rcu4D3GNce6DHh9oGbgf8O/E+Iw8ffv01XPXr10Pl9uon+/P37/kEfc8EptLTaI09kAAUymDBw9WB8GUKVOGm+hhWVi5auV/mnjzxk3JnSu34EkVqXz58jq9AItI2KAirVq3Cn/6mzhxoj6hwnSNNGbMGMmWLZseYxknEupEMpZbRnQixFNk9x7d9Xv8GTVqlDx6GLb88zV1wbKD1Rlhg4WEKRd6LZ48kdKnT6/vtWrVkj/++EOtATiHJ/cBnw2QTJky6RQNplfq16+vec+fP6/vYYNb+PXzF8xXXq1bt9bv8MfT01OPA+4EiCUyhA1Imh9TZ0jgAbnDBqZwP43AoED9LkGCBLHGWCuI8OfChQv6CQ6wSH5+fhI2+Mrnn3+un/Hn/v37kjLVs37D56NHj0qVKlVwqAlTOK/rI+N+iMh06dKlakHy2eAjyZI9c7o1lizjnjDuxU/7fyrg8WIypo8i3j9G3+XInkOzz5w1U+8LWCuMVLBQQT28c/eOGPlfd58a1+H94cOHatEJU2T1dGT3xZq1azRfjx499B1/MmfOrPfSrdu39NyZM2f0s3Ff4GSatM8cd437WDPyDwmYhAAVE5N0hD2JMWfOHB0QsAoDvh1GwiCJdMrvlOCHHNMcUFYqVKygCsjevXsl7AlUYMpftOjZtEiZ0mWkYsWK8sXnX0ihQoUkceJnt+y5888GYUNRQLnnLzwb8DNnyYyPYgwsWbNm1c/406ZNG32Fnwg7iKwu+KsgYYrESMa5bNmfKUXVqlbTr3y3+Qp8BNB2TOUgGUtes+d4NjAbsjdp0kS/x5/9+/dL/vz5w9uHc6dOncKbeOb3VJ8ZHL9OBrQ3S5YsyhZ5/S/5400VPT0I+2MoL1mzZJXVq1fraWszNuoy3sMVk39WmaBdmBJxcXExsmhbGzRooJ8fPHjw3OCMk1BmkCKTNSJTw2cFiq6RDN8L3BPz5s3T0/DLeVkyWBmKL/JEVHahLGJaCP0TMZ08cVI/QrnAqjSk192nmuGfPxcuPlPi4HeEFNl9YSxZzp07t+bHH8iEf2tYno0UZpGU5s2ahyvsOAc2UA4NPxacYyIBsxBIaBZBKId9EMBTOnwFYOm4dfOW3L51O/y1ZvWzpztjCWiqVKk0VsPmTZtl/LjxCsCwKri5ueln+Ji0a9dOihUrpoO28QQaZibX7zNneqaE4INxDoMukvGkiHpelyKryxhYMegbyTiXLeszxQRP1VCyFi5cKKO+HyUDBgwQd3d3zW7kNSwGhpzGZ2SClQV+AMYTLPwy4NuBOrEqxSjjdTJg8Db8IlBm2JQG3sSQEcew3EBpgg9OZO025IwpY0NBhOyG5cdQ6CATBlG8cubIiY9y0f+ZImgMzjhnqawRmcJigQRrDBIGccPvAn4/zi7Oeh4yGQm+J7BaIBmswMtIZ8+d1UMoNvBLQTp0+N/YLHC0hSMzrFRJkyYNvydfx1AL+edP2PSSHhkWmcjuCzf3Z/9OTpw4EV4MfE6Q3mj8hvrSgG3efM/8ZXB+3759yqJ79+5qKUIARFg5mUjALASomJilJ+xEjhm/z9Cn+8GDBv/HPG4s+zQGBwxYCDw2depUXYkBBP/74X9KAnkxIAwbNkywgmL9+vW6KqJosaLqMAkHQTzx4cffSCgP1xgDCQZ5/CjDIoKVHq9KkdVlDEaGJQblGNaZiNYYPPHDqRNyhc3nh1dnPGUbT96QHYO04QyKjJgmQvrggw90hQUcHhH3ZdzYcWpZsEQGWCKMwR1lvVgvzkFZMywEkbXbWowNywMGWcPSFJGbYSUwlBVjkMRSYTgXY7WRJbK+yLRgwWdTKnAoxn2GIG+wYsAhG6l2rdr6jhVAUJZxn3kV8BJjYDfun8WLF4ffP0Zb0MfovzB/GR3kv/3uW0E+rH5BGvnNSH23hKFm/OePYU0z7pXI7ouGDRrqlX369tEAhkOGDpEhQ4ZImP+OWhgN3rBirlu3TiPKwgkWSnSvT545vzZs1FDSeaQL75uI8vCYBOKDABWT+KBup3XihxzxF0qVKqUrcV5sJnw/wpwcdVUNlvji6R6mbsSraFC/gfT8sKcu3YQvAZQLrKBwSuIkXbt21dUpWOmDpZt4esYAmzPnsydso56zYYGp8uTJY3yUju901HJavtlSn5TxZPiyFFldxlNsxKdewxqB+Xwj1ahRQw/BwPBpwAlj4DUGY8ge0bKBPGFOwTqNhadtTH9ByYCFqWHDZwNPZDLgSR9KWEQrgzGVE1FGmPVz5cqFKuOMMZQqDIRYXYVluEiGVQvHiNeBZFh2MDUCJQ8rrKBUYAVMZH30MqZhDqFquUO03TDHaVm6ZKmUKFlCp8xQH6Z4cB6raLA0G1NvUAyx4goJ16PeFi1bqJKI+wfKb5gTbrjSPfqn0SorlmcjH1hjStLwS7HkPtXK/vljWASNeyWy+wJKFlYTYcoHy+qh5P/000+6AglFGsop/q3UrVdXEFEWK7MwjWdYfIKDg/XeCX1Mq0nEvuBx/BFggLX4Y+/QNSMmBJSMS/6Xwjkg3gJij2DwhDOskWBqd3Z2fs4nwfgusndMLcE0jwHGkhSTuiwpP7I8sBZA3oiKTWTXWOP7mLQ7qowtlRcyYRoOCk3EFFVZkR/K7MscXFEuFN5bt25pHsOHyajP0rZhughyRrTgGWVY4z2y+wJKE/bYeXHaEtYfxL/xO+Wnjti4twyFxJALDxTqfPyP07hxnu8kEF8EEsdXxazXsQlgygFz/li9gyc4mLwRZwKm8YhKCSgZvhrRIebk5CR4WZpiUpeldbwuHwbGFwfH1+W31ncxaXdUGVsq86tketX5V5UbWX4oFPA5eVmytG0RHXlfVk5Mz0V2X0DpelEpQZ2GnxBWh2HqKeL0oSHTq84b3/OdBOKaABWTuCbO+pTA8OHD9QnVe5m3YMklpjYQCKxTx04kRAIkYCUCqdxSSViMnZcqJFaqgsWQgNUJcCrH6khZIAmQAAmQAAmQQHQJPD95G91SeB0JkAAJkAAJkAAJWIEAFRMrQGQRJEACJEACJEAC1iFAxcQ6HFkKCZAACZAACZCAFQhQMbECRBZBAiRAAiRAAiRgHQJUTKzDkaWQAAmQAAmQAAlYgQAVEytAZBEkQAIkQAIkQALWIUDFxDocWQoJkAAJkAAJkIAVCFAxsQJEFkECJEACJEACJGAdAlRMrMORpZAACZAACZAACViBABUTK0BkESRAAiRAAiRAAtYhQMXEOhxZCgmQAAmQAAmQgBUIUDGxAkQWQQIkQAIkQAIkYB0CVEysw5GlkAAJkAAJkAAJWIEAFRMrQGQRJEACJEACJEAC1iHwf/V0c6KllINJAAAAAElFTkSuQmCC",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 500,
       "width": 500
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import Image, display\n",
    "display(Image(filename='../assets/RL/1.png', width=500, height=500))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key Terms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ States and Observations\n",
    "\n",
    "- A **state** $s$: a complete description of the state of the world. I.e. the status of the agent with respect to the environment at an instant in time.\n",
    "- **State space**: the set of all states $S$\n",
    "<br><br>\n",
    "States and observations are represented by a vector, matrix, or tensor. For example, a visual observation can be represented by the RGB matrix of its pixel values; the state of a robot can be represented by its joint angles and velocities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Action Spaces\n",
    "\n",
    "- **Action** $a$: For each state, the agent can perform an action. This will determine the next state. Different environments allow different types of actions.\n",
    "\n",
    "- **Action space of the set** $A$: The set of all possible actions in a given environment. The action space is dependent on the state (different situations can have different possible actions)\n",
    "<br><br>\n",
    "- **Discrete action spaces**: Some environments, like video games, only have a FINITE number of moves are available to the agent. \n",
    "- **Continuous action spaces**: Other environments, like where the agent controls a robot in a physical world, have continuous possible actions. In continuous spaces, actions are real-valued vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Policies\n",
    "\n",
    "A policy is a rule used by an agent to decide what actions to take. I.e., a policy tells the agent what actions to take at a given state.\n",
    "<br><br>\n",
    "➤ Deterministic Policies\n",
    "$$a_t = \\mu(s_t)$$ \n",
    "- where $\\mu$ is the **deterministic** policy function.\n",
    "- Deterministic policies always select the **same action** for a given state.\n",
    "\n",
    "Conditional probabilities can describe state transitions. For example,\n",
    "\n",
    "$$p(s_2 | s_1, a_2) = 1$$\n",
    "\n",
    "This means: If I am currently in $s_1$ and take action $a_2$, what is the **probability** $p$ that I will move to $s_2$? This probability is $1$.\n",
    "\n",
    "This is deterministic because the probability is fixed at $1$. But, if it is a probability distribution, it is a **stochastic** case.\n",
    "<br><br>\n",
    "\n",
    "➤ Stochastic Policies\n",
    "$$a_t \\sim \\pi(\\cdot | s_t)$$\n",
    "- where $\\pi$ denotes the **stochastic** policy function. \n",
    "    - $\\pi$ is the conditional probability that determines the likelihood of taking any action in any given state.\n",
    "- Stochastic policies select an action **randomly** based on a **probability distribution**. Even if the agent is in the same state multiple times, it might take **different actions** each time.\n",
    "- Two key computations are centrally important to use and train stochastic policies:\n",
    "    - sampling actions from the policy,\n",
    "    - computing log likelihoods of particular actions: $log\\pi_\\theta (a | s)$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Reward\n",
    "\n",
    "The goal of RL is to select a policy which maximizes **expected return** when the agent acts according to it.\n",
    "\n",
    "- **Reward**: a scalar an agent gets after taking an action.\n",
    "    - **Positive** reward: encouragement to take such actions\n",
    "    - **Negative** reward: punishment to take such actions\n",
    "    - **0** value: No encouragement or punishment\n",
    "    - The absolute value of a reward does not matter. Instead, it is the relative value that matters.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Trajectory and Return\n",
    "\n",
    "- **Trajectory**: a state-action-reward chain <br>\n",
    "    - Finite trajectories are called **episodes** or **trials**.\n",
    "\n",
    "\n",
    "ex. agent is in state $s_1$, takes action $a_2$, receives reward $r = 0$. agent is now in state $s_2$, takes action $a_3$, etc ...\n",
    "<br><br><br>\n",
    "\n",
    "➤ Return and Discounted Return\n",
    "\n",
    "- **Return** $R(s,a)$: the return of a trajectory is the **sum of all the rewards** collected along the trajectory\n",
    "    - can be used to evaluate whether a policy is good or not\n",
    "\n",
    "- **Discounted return**: the sum of all the reward values multiplied by coefficients (discounts)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Value Functions\n",
    "\n",
    "Value, as in, the expected return if an agent starts in a particular state (or state-action pair), and then acts according to a particular policy forever after. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Markov Property\n",
    "\n",
    "RL is **markovian**: only the **current** state determines the next state. Past states do not matter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ➤ Markov decision process (MDP)\n",
    "\n",
    "Key elements of MDP:\n",
    "\n",
    "#### - Sets:\n",
    "- State: the set of states $S$\n",
    "- Action: the set of actions $A(s)$ is associated for state $s \\in S$\n",
    "- Reward: the set of rewards $R(s,a)$\n",
    "\n",
    "#### - Probability distribution:\n",
    "- State transition probability: at state $s$, taking action $a$, the probability to transit to state $s'$ is $p(s'|s,a)$\n",
    "- Reward probability: at state $s$, taking action $a$, the probability to get reward $r$ is $p(r|s,a)$\n",
    "\n",
    "#### - Policy:\n",
    "at state $s$, the probability to choose action $a$ is $\\pi(a|s)$\n",
    "\n",
    "#### - Markov property:\n",
    "the memoryless property. i.e. previous states and actions are irrelevant to the next ones.\n",
    "$$p(s_{t+1}|a_{t+1}, s_{t}, ..., a_1, s_0) = p(s_{t+1}|a_{t+1},s_t)$$\n",
    "$$p(r_{t+1}|a_{t+1}, s_{t}, ..., a_1, s_0) = p(r_{t+1}|a_{t+1},s_t)$$\n",
    "\n",
    "<br><br>\n",
    "The Markov decision process broken down: 'Markov' corresponds to the Markov property, 'decision' corresponds to the policy because making a decision requires a policy, 'process' corresponds to the transition from one state to another based on the actions taken. These aspects are defined by sets and probability distributions.\n",
    "\n",
    "Once the policy is given, a Markov decision process becomes a Markov process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recommended Resources:\n",
    "- [OpenAI Spinning Up](https://spinningup.openai.com/en/latest/spinningup/rl_intro.html)\n",
    "- [Mathematical Foundations of Reinforcement Learning Playlist](https://www.youtube.com/watch?v=ZHMWHr9811U&list=PLEhdbSEZZbDaFWPX4gehhwB9vJZJ1DNm8&index=1&ab_channel=WINDYLab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ML-from-scratch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
